Robust Recognition of Complex Gestures for Natural Human-Robot Interaction

Robots coexisting with humans in everyday environments should be able to interact with them in an intuitive way. This requires that the robots are able to recognize typical gestures performed by humans such as pointing gestures, waving, or head shaking/nodding. We present a system that is able to spot and recognize complex, parameterized gestures from data of a monocular camera. To represent people, we locate their faces and hands using trained classifiers and track them over time. We use few, expressive features extracted from this compact representation as input to hidden Markov models (HMMs). First, we segment the gestures into distinct phases and train HMMs for each phase separately. Then, we construct composed HMMs, which consist of the individual phase-HMMs. Once a specific phase is recognized, we estimate the parameter of a gesture such as the target of a pointing gesture. As we demonstrate in the experiments, our system is able to robustly spot and recognize a variety of complex gestures.